A mask-guided attention deep learning model for COVID-19 diagnosis based on an integrated CT scan images database

The global extent of COVID-19 mutations and the consequent depletion of hospital resources highlighted the necessity of effective computer-assisted medical diagnosis. COVID-19 detection mediated by deep learning models can help diagnose this highly contagious disease and lower infectivity and mortality rates. Computed tomography (CT) is the preferred imaging modality for building automatic COVID-19 screening and diagnosis models. It is well-known that the training set size significantly impacts the performance and generalization of deep learning models. However, accessing a large dataset of CT scan images from an emerging disease like COVID-19 is challenging. Therefore, data efficiency becomes a significant factor in choosing a learning model. To this end, we present a multi-task learning approach, namely, a mask-guided attention (MGA) classifier, to improve the generalization and data efficiency of COVID-19 classification on lung CT scan images. The novelty of this method is compensating for the scarcity of data by employing more supervision with lesion masks, increasing the sensitivity of the model to COVID-19 manifestations, and helping both generalization and classification performance. Our proposed model achieves better overall performance than the single-task (without MGA module) baseline and state-of-the-art models, as measured by various popular metrics.

Attentive Multi-Task Prediction of Atmospheric Particulate Matter: Effect of the COVID-19 Pandemic

Air pollution, especially the continual increase in atmospheric particulate matter (PM), is a global environmental challenge. To reduce the PM concentration, a remarkable amount of machine learning-based research has been proposed. However, increasing the accuracy of the predictions and providing clear interpretations of the predictions are challenging. In particular, no studies have addressed models that predict and interpret PM before and during the COVID-19 pandemic. In this paper, we present a two-step predictive and explainable model to obtain insights into reducing PM. We first use attentive multi-task learning to predict the air quality of cities. To accurately predict the concentration of particles with sizes of similar to 10 mu m or similar to 2.5 mu m (PM10 and PM2.5, respectively), we demonstrate a performance difference between single-task and multi-task learning, as well as among the state-of-the art methods. The proposed attentive model with multi-task learning outperformed the others in terms of accuracy performance. We then used Shapley additive explanations, a representative explainable artificial intelligence framework, to interpret and determine the significance of features for predicting PM10 and PM2.5. We demonstrated the superiority of the proposed approach in predicting and explaining both PM10 and PM2.5 concentrations, and observed a statistically significant difference in air pollution before and during the COVID-19 pandemic.

A joint segmentation and classification framework for COVID-19 infection segmentation and detection from chest CT images

In COVID19 management, CT images are used as noninvasive diagnostic tools for screening and disease monitoring. Segmentation of infections provides valuable visual interpretations in the process of prognosis and decision making. Segmentation of COVID19 infection from chest CT images is challenging due to the presence of multiple infection types and complex morphological patterns. This paper presents a novel multi task learning framework for COVID19 infection segmentation and detection. The proposed model called DB-YNet, is built on YNet architecture with a dense bottleneck and attention based UNet backbone. This model is trained and tested with standard datasets, demonstrating superior segmentation and classification metrics. It achieves a Dice score of 0.9923 for segmentation, and classification accuracies of 0. 0.9875 and 0.9961 under binary and multi class classifications respectively. In COVID19 triage, DB-YNet is a promising tool to assist physicians in the early identification of COVID19 infected patients for quick clinical interventions. © 2023 Wiley Periodicals LLC.

COVID19-MLSF: A multi-task learning-based stock market forecasting framework during the COVID-19 pandemic.

The sudden outbreak of COVID-19 has dramatically altered the state of the global economy, and the stock market has become more volatile and even fallen sharply as a result of its negative impact, heightening investors' apprehension regarding the correlation between unexpected events and stock market volatility. Additionally, internal and external characteristics coexist in the stock market. Existing research has struggled to extract more effective stock market features during the COVID-19 outbreak using a single time-series neural network model. This paper presents a framework for multitasking learning-based stock market forecasting (COVID-19-MLSF), which can extract the internal and external features of the stock market and their relationships effectively during COVID-19.The innovation comprises three components: designing a new market sentiment index (NMSI) and COVID-19 index to represent the external characteristics of the stock market during the COVID-19 pandemic. Besides, it introduces a multi-task learning framework to extract global and local features of the stock market. Moreover, a temporal convolutional neural network with a multi-scale attention mechanism is designed (MA-TCN) alongside a Multi-View Convolutional-Bidirectional Recurrent Neural Network with Temporal Attention (MVCNN-BiLSTM-Att), adjusting the model to account for the changing status of COVID-19 and its impact on the stock market. Experiments indicate that our model achieves superior performance both in terms of predicting the accuracy of the China CSI 300 Index during the COVID-19 period and in terms of sing market trading.

MTSS-AAE: Multi-task semi-supervised adversarial autoencoding for COVID-19 detection based on chest X-ray images.

Efficient diagnosis of COVID-19 plays an important role in preventing the spread of the disease. There are three major modalities to diagnose COVID-19 which include polymerase chain reaction tests, computed tomography scans, and chest X-rays (CXRs). Among these, diagnosis using CXRs is the most economical approach; however, it requires extensive human expertise to diagnose COVID-19 in CXRs, which may deprive it of cost-effectiveness. The computer-aided diagnosis with deep learning has the potential to perform accurate detection of COVID-19 in CXRs without human intervention while preserving its cost-effectiveness. Many efforts have been made to develop a highly accurate and robust solution. However, due to the limited amount of labeled data, existing solutions are evaluated on a small set of test dataset. In this work, we proposed a solution to this problem by using a multi-task semi-supervised learning (MTSSL) framework that utilized auxiliary tasks for which adequate data is publicly available. Specifically, we utilized Pneumonia, Lung Opacity, and Pleural Effusion as additional tasks using the ChesXpert dataset. We illustrated that the primary task of COVID-19 detection, for which only limited labeled data is available, can be improved by using this additional data. We further employed an adversarial autoencoder (AAE), which has a strong capability to learn powerful and discriminative features, within our MTSSL framework to maximize the benefit of multi-task learning. In addition, the supervised classification networks in combination with the unsupervised AAE empower semi-supervised learning, which includes a discriminative part in the unsupervised AAE training pipeline. The generalization of our framework is improved due to this semi-supervised learning and thus it leads to enhancement in COVID-19 detection performance. The proposed model is rigorously evaluated on the largest publicly available COVID-19 dataset and experimental results show that the proposed model attained state-of-the-art performance.

General Short-Term Load Forecasting Based on Multi-Task Temporal Convolutional Network in COVID-19

The spread of the global COVID-19 epidemic has resulted in significant shifts in electricity consumption compared to regular days. It is unknown if standard single-task, single-indicator load forecasting algorithms can accurately reflect COVID-19 load patterns. Power practitioners urgently want a simple, efficient, and accurate solution for anticipating reliable load. In this paper, we first propose a unique collaborative TCN-LSTM-MTL short-term load forecasting model based on mobility data, temporal convolutional networks, and multi-task learning. The addition of the parameter sharing layers and the structure with residual convolution improves the data input diversity of the forecasting model and enables the model to obtain a wider time series receptive field. Then, to demonstrate the usefulness of the mobility optimized TCN-LSTM-MTL, tests were conducted in three levels and twelve base regions using 19 different benchmark models. It is capable of controlling predicting mistakes to within 1% in the majority of tasks. Finally, to rigorously explain the model, the Shapley additive explanations (SHAP) visual model interpretation technology based on game theory is introduced. It examines the TCN-LSTM-MTL model's internal mechanism at various time periods and establishes the validity of the mobility indicators as well as the asynchronous relationship between indicator significance and real contribution.

Stance Detection on Tweets with Multi-task Aspect-based Sentiment: A Case Study of COVID-19 Vaccination

Public opinion analyses on Twitter conducted based on sentiment analysis cannot identify the author’s stance regarding agreement or disagreement with a given target. Stance detection determines whether the author of a text is in favor, against, or neutral towards a target. However, stance detection based on text-only is less representative opinion, especially on a tweet, which is a short text with slightly contextual information. Therefore, more information is needed to represent the author's stance better. In previous research, most research on stance detection was carried out using simple sentiment information to measure the support to target. This study addresses multi-task aspect-based sentiment analysis (ABSA) and social features for stance detection based on deep learning models of BiGRU-BERT on tweets. Our contribution combines aspect-based sentiment information with features based on textual and contextual information that does not emerge directly from Twitter texts. ABSA approach can provide more accurate sentiment information at aspect level on tweets, which is possible contains multiple issues discussed. Aspect information on tweets can reflect the issue that influences the author’s stance toward a target. Multi-task learning was applied to help improve the generalization performance of ABSA with simultaneous processes. We extracted social attributes and online behavioral features for contextual information. Since same community tends to have the same opinion towards a target, we applied a community detection task and combine with the Twitter social attributes. The proposed method has significantly improved evaluation metrics (>10%) than textual features only for stance detection on tweets © 2022. International Journal of Intelligent Engineering and Systems.All Rights Reserved

Multi-task learning for predicting SARS-CoV-2 antibody escape.

The coronavirus pandemic has revolutionized our world, with vaccination proving to be a key tool in fighting the disease. However, a major threat to this line of attack are variants that can evade the vaccine. Thus, a fundamental problem of growing importance is the identification of mutations of concern with high escape probability. In this paper we develop a computational framework that harnesses systematic mutation screens in the receptor binding domain of the viral Spike protein for escape prediction. The framework analyzes data on escape from multiple antibodies simultaneously, creating a latent representation of mutations that is shown to be effective in predicting escape and binding properties of the virus. We use this representation to validate the escape potential of current SARS-CoV-2 variants.

Self-supervision and Multi-task Learning: Challenges in Fine-Grained COVID-19 Multi-class Classification from Chest X-rays

Quick and accurate diagnosis is of paramount importance to mitigate the effects of COVID-19 infection, particularly for severe cases. Enormous effort has been put towards developing deep learning methods to classify and detect COVID-19 infections from chest radiography images. However, recently some questions have been raised surrounding the clinical viability and effectiveness of such methods. In this work, we investigate the impact of multi-task learning (classification and segmentation) on the ability of CNNs to differentiate between various appearances of COVID-19 infections in the lung. We also employ self-supervised pre-training approaches, namely MoCo and inpainting-CXR, to eliminate the dependence on expensive ground truth annotations for COVID-19 classification. Finally, we conduct a critical evaluation of the models to assess their deploy-readiness and provide insights into the difficulties of fine-grained COVID-19 multi-class classification from chest X-rays. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Prediction of Potential Commercially Available Inhibitors against SARS-CoV-2 by Multi-Task Deep Learning Model.

The outbreak of COVID-19 caused millions of deaths worldwide, and the number of total infections is still rising. It is necessary to identify some potentially effective drugs that can be used to prevent the development of severe symptoms, or even death for those infected. Fortunately, many efforts have been made and several effective drugs have been identified. The rapidly increasing amount of data is of great help for training an effective and specific deep learning model. In this study, we propose a multi-task deep learning model for the purpose of screening commercially available and effective inhibitors against SARS-CoV-2. First, we pretrained a model on several heterogenous protein-ligand interaction datasets. The model achieved competitive results on some benchmark datasets. Next, a coronavirus-specific dataset was collected and used to fine-tune the model. Then, the fine-tuned model was used to select commercially available drugs against SARS-CoV-2 protein targets. Overall, twenty compounds were listed as potential inhibitors. We further explored the model interpretability and exhibited the predicted important binding sites. Based on this prediction, molecular docking was also performed to visualize the binding modes of the selected inhibitors.

Leveraging Discriminative Cues for Masked Face Recognition in Post COVID World

The post COVID world has completely disrupted our lifestyle, where wearing a mask is necessary to protect ourselves and others from contracting the virus. However, face masks have proved to be challenging for facial biometric systems, in the sense that these systems do not work as expected when wearing masks as nearly half of the face is covered, thus reducing discriminative features that the model can leverage. Most of the existing frameworks rely on the entire face as the input, but as the face is covered, these frameworks do not perform up to the mark. Moreover, training another facial recognition system with mask images is challenging as the availability of datasets is limited, both qualitatively and quantitatively. In this paper, we propose a framework that shows better results without significant training. In the proposed work, firstly we extracted the face using SSD, then by obtaining Facial Landmarks for utilizing the cues from other dis-criminative parts for facial recognition. The proposed framework is able to out-perform other frameworks on facial mask images and also found ~4.5% increment in accuracy. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Analysis of mobility based COVID-19 epidemic model using Federated Multitask Learning.

Aggregating a massive amount of disease-related data from heterogeneous devices, a distributed learning framework called Federated Learning(FL) is employed. But, FL suffers in distributing the global model, due to the heterogeneity of local data distributions. To overcome this issue, personalized models can be learned by using Federated multitask learning(FMTL). Due to the heterogeneous data from distributed environment, we propose a personalized model learned by federated multitask learning (FMTL) to predict the updated infection rate of COVID-19 in the USA using a mobility-based SEIR model. Furthermore, using a mobility-based SEIR model with an additional constraint we can analyze the availability of beds. We have used the real-time mobility data sets in various states of the USA during the years 2020 and 2021. We have chosen five states for the study and we observe that there exists a correlation among the number of COVID-19 infected cases even though the rate of spread in each case is different. We have considered each US state as a node in the federated learning environment and a linear regression model is built at each node. Our experimental results show that the root-mean-square percentage error for the actual and prediction of COVID-19 cases is low for Colorado state and high for Minnesota state. Using a mobility-based SEIR simulation model, we conclude that it will take at least 400 days to reach extinction when there is no proper vaccination or social distance.

Zero-Shot and Few-Shot Classification of Biomedical Articles in Context of the COVID-19 Pandemic

MeSH (Medical Subject Headings) is a large thesaurus created by the National Library of Medicine and used for fine-grained indexing of publications in the biomedical domain. In the context of the COVID-19 pandemic, MeSH descriptors have emerged in relation to articles published on the corresponding topic. Zero-shot classification is an adequate response for timely labeling of the stream of papers with MeSH categories. In this work, we hypothesise that rich semantic information available in MeSH has potential to improve BioBERT representations and make them more suitable for zero-shot/few-shot tasks. We frame the problem as determining if MeSH term definitions, concatenated with paper s are valid instances or not, and leverage multi-task learning to induce the MeSH hierarchy in the representations thanks to a seq2seq task. Results establish a baseline on the MedLine and LitCovid datasets, and probing shows that the resulting representations convey the hierarchical relations present in MeSH. © 2021 Copyright for this paper by its authors.

Explainable multi-instance and multi-task learning for COVID-19 diagnosis and lesion segmentation in CT images.

Coronavirus Disease 2019 (COVID-19) still presents a pandemic trend globally. Detecting infected individuals and analyzing their status can provide patients with proper healthcare while protecting the normal population. Chest CT (computed tomography) is an effective tool for screening of COVID-19. It displays detailed pathology-related information. To achieve automated COVID-19 diagnosis and lung CT image segmentation, convolutional neural networks (CNNs) have become mainstream methods. However, most of the previous works consider automated diagnosis and image segmentation as two independent tasks, in which some focus on lung fields segmentation and the others focus on single-lesion segmentation. Moreover, lack of clinical explainability is a common problem for CNN-based methods. In such context, we develop a multi-task learning framework in which the diagnosis of COVID-19 and multi-lesion recognition (segmentation of CT images) are achieved simultaneously. The core of the proposed framework is an explainable multi-instance multi-task network. The network learns task-related features adaptively with learnable weights, and gives explicable diagnosis results by suggesting local CT images with lesions as additional evidence. Then, severity assessment of COVID-19 and lesion quantification are performed to analyze patient status. Extensive experimental results on real-world datasets show that the proposed framework outperforms all the compared approaches for COVID-19 diagnosis and multi-lesion segmentation.

Multi-branch fusion auxiliary learning for the detection of pneumonia from chest X-ray images.

Lung infections caused by bacteria and viruses are infectious and require timely screening and isolation, and different types of pneumonia require different treatment plans. Therefore, finding a rapid and accurate screening method for lung infections is critical. To achieve this goal, we proposed a multi-branch fusion auxiliary learning (MBFAL) method for pneumonia detection from chest X-ray (CXR) images. The MBFAL method was used to perform two tasks through a double-branch network. The first task was to recognize the absence of pneumonia (normal), COVID-19, other viral pneumonia and bacterial pneumonia from CXR images, and the second task was to recognize the three types of pneumonia from CXR images. The latter task was used to assist the learning of the former task to achieve a better recognition effect. In the process of auxiliary parameter updating, the feature maps of different branches were fused after sample screening through label information to enhance the model's ability to recognize case of pneumonia without impacting its ability to recognize normal cases. Experiments show that an average classification accuracy of 95.61% is achieved using MBFAL. The single class accuracy for normal, COVID-19, other viral pneumonia and bacterial pneumonia was 98.70%, 99.10%, 96.60% and 96.80%, respectively, and the recall was 97.20%, 98.60%, 96.10% and 89.20%, respectively, using the MBFAL method. Compared with the baseline model and the model constructed using the above methods separately, better results for the rapid screening of pneumonia were achieved using MBFAL.

Leveraging OpenStreetMap and Multimodal Remote Sensing Data with Joint Deep Learning for Wastewater Treatment Plants Detection.

Humans rely on clean water for their health, well-being, and various socio-economic activities. During the past few years, the COVID-19 pandemic has been a constant reminder of about the importance of hygiene and sanitation for public health. The most common approach to securing clean water supplies for this purpose is via wastewater treatment. To date, an effective method of detecting wastewater treatment plants (WWTP) accurately and automatically via remote sensing is unavailable. In this paper, we provide a solution to this task by proposing a novel joint deep learning (JDL) method that consists of a fine-tuned object detection network and a multi-task residual attention network (RAN). By leveraging OpenStreetMap (OSM) and multimodal remote sensing (RS) data, our JDL method is able to simultaneously tackle two different tasks: land use land cover (LULC) and WWTP classification. Moreover, JDL exploits the complementary effects between these tasks for a performance gain. We train JDL using 4,187 WWTP features and 4,200 LULC samples and validate the performance of the proposed method over a selected area around Stuttgart with 723 WWTP features and 1,200 LULC samples to generate an LULC classification map and a WWTP detection map. Extensive experiments conducted with different comparative methods demonstrate the effectiveness and efficiency of our JDL method in automatic WWTP detection in comparison with single-modality/single-task or traditional survey methods. Moreover, lessons learned pave the way for future works to simultaneously and effectively address multiple large-scale mapping tasks (e.g., both mapping LULC and detecting WWTP) from multimodal RS data via deep learning.

Recurrent Multi-task Graph Convolutional Networks for COVID-19 Knowledge Graph Link Prediction

Knowledge graphs (KGs) are a way to model data involving intricate relations between a number of entities. Understanding the information contained in KGs and predicting what hidden relations may be present can provide valuable domain-specific knowledge. Thus, we use data provided by the 5th Annual Oak Ridge National Laboratory Smoky Mountains Computational Sciences Data Challenge 2 as well as auxiliary textual data processed with natural language processing techniques to form and analyze a COVID-19 KG of biomedical concepts and research papers. Moreover, we propose a recurrent graph convolutional network model that predicts both the existence of novel links between concepts in this COVID-19 KG and the time at which the link will form. We demonstrate our model’s promising performance against several baseline models. The utilization of our work can give insights that are useful in COVID-19-related fields such as drug development and public health. All code for our paper is publicly available at https://github.com/RemingtonKim/SMCDC2021. © 2022, Springer Nature Switzerland AG.

A Vision-Based System for In-Sleep Upper-Body and Head Pose Classification.

Sleep quality is known to have a considerable impact on human health. Recent research shows that head and body pose play a vital role in affecting sleep quality. This paper presents a deep multi-task learning network to perform head and upper-body detection and pose classification during sleep. The proposed system has two major advantages: first, it detects and predicts upper-body pose and head pose simultaneously during sleep, and second, it is a contact-free home security camera-based monitoring system that can work on remote subjects, as it uses images captured by a home security camera. In addition, a synopsis of sleep postures is provided for analysis and diagnosis of sleep patterns. Experimental results show that our multi-task model achieves an average of 92.5% accuracy on challenging datasets, yields the best performance compared to the other methods, and obtains 91.7% accuracy on the real-life overnight sleep data. The proposed system can be applied reliably to extensive public sleep data with various covering conditions and is robust to real-life overnight sleep data.

Design, implementation and interpretation of algorithms to predict the progression of Alzheimer's disease

Alzheimer's Disease (AD) is a serious public health issue that results in significant social and financial burdens on the individuals and communities impacted. In order to tackle this public health crisis it is critical that the clinical and computational research communities collaborate to identify possible causes of this progressive memory disease. Close collaboration between these two communities has the potential to result in promising therapeutic treatments for AD and other health conditions. This dissertation presents a collection of algorithms and associated derivations designed to predict the progression of AD using multi-task and structured regularization techniques, clustering membership by way of nonnegative matrix factorization, and COVID-19 clinical outcome prediction using multi-instance learning methods. This work presents novel algorithms for handling multimodal and longitudinal data and details approaches for multitask and multi-instance learning techniques that can be applied in other fields. Extensive discussions on algorithm predictive performance, interpretability, and implementation are provided for each method and are designed to serve as a framework for future research. (PsycInfo Database Record (c) 2022 APA, all rights reserved)

MultiR-Net: A Novel Joint Learning Network for COVID-19 segmentation and classification.

The outbreak of COVID-19 has caused a severe shortage of healthcare resources. Ground Glass Opacity (GGO) and consolidation of chest CT scans have been an essential basis for imaging diagnosis since 2020. The similarity of imaging features between COVID-19 and other pneumonia makes it challenging to distinguish between them and affects radiologists' diagnosis. Recently, deep learning in COVID-19 has been mainly divided into disease classification and lesion segmentation, yet little work has focused on the feature correlation between the two tasks. To address these issues, in this study, we propose MultiR-Net, a 3D deep learning model for combined COVID-19 classification and lesion segmentation, to achieve real-time and interpretable COVID-19 chest CT diagnosis. Precisely, the proposed network consists of two subnets: a multi-scale feature fusion UNet-like subnet for lesion segmentation and a classification subnet for disease diagnosis. The features between the two subnets are fused by the reverse attention mechanism and the iterable training strategy. Meanwhile, we proposed a loss function to enhance the interaction between the two subnets. Individual metrics can not wholly reflect network effectiveness. Thus we quantify the segmentation results with various evaluation metrics such as average surface distance, volume Dice, and test on the dataset. We employ a dataset containing 275 3D CT scans for classifying COVID-19, Community-acquired Pneumonia (CAP), and healthy people and segmented lesions in pneumonia patients. We split the dataset into 70% and 30% for training and testing. Extensive experiments showed that our multi-task model framework obtained an average recall of 93.323%, an average precision of 94.005% on the classification test set, and a 69.95% Volume Dice score on the segmentation test set of our dataset.